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ABSTRACT 


Biometric based personal authentication is playing a vital role in various 
security based applications. This paper presents the effective fusion of 
fingerprint, palmprint and iris traits at decision level. Combining different 
traits at the decision level is a challenging task due to less information 
available at this level. The focus of the work is to examine the performance 
of multimodal biometrics at decision level fusion in three different i.e. serial, 
parallel and hierarchical modes of operation. Serial mode is performed by 
taking unimodals serially while parallel mode of operation is carried out by 
processing all modals simulatenously using Majority Voting Rule and the 
hierarchical mode of operation is performed with proper combination of 
traits in parallel and serial mode using AND and OR rule. The experiments 
are performed on 100 different users from publically available FVC2006 
fingerprint database, CASIA V1 palmprint database and IITD iris database. 
The experimental results suggest that proper fusion of different traits in 
hierarchical way can give best performance even at decision level fusion as 


compared to serial and parallel mode of operation. 
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1, INTRODUCTION 

Automatic personal authentication 1s becoming an essential part in large variety of security 
applications. Traditional authentication methods are based on use of key, ID cards, PIN or password. 
However, with this method there are lots of chances of breaking the security. Biometrics is a pattern 
recognition system which authenticates the person based on physiological traits like palmprint, iris, 
fingerprint, retina, face, finger-vein, fingerknuckle or behavioral traits like signature, voice, gait etc [1, 2]. 
These biometric features cannot be misplaced or stolen hence biometrics is frontier to traditional methods of 
personal authentication. The biometric system which uses single trait for authentication is a unimodal 
biometrics. However unimodal biometrics faces some problems like spoofing, interclass similarity, intra- 
class variations, universality. Also no single biometric characteristic can satisfy all the performance 
requirements of the system [3]. To alleviate the problems of unimodal biometrics; biometric researchers are 
now moving towards multimodal biometrics. 

Multimodal biometrics makes the use of multiple traits for authentication purpose [4]. Multimodal 
biometrics integrates the information from different traits at various levels. Proper integration of the 
information which is gained from different modals is very important to get best system performance [5]. 
Information from different modals can be merged at Feature, Matching score or Decision level. In features 
level fusion, features of different traits are merged and form a new feature set which is used for 
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authentication. In matching score level fusion, matching scores of different modals are combined after 
normalization of the scores. In decision level fusion, each modal gives its decision and finally by applying 
rules final decision 1s made. Feature level fusion is pre-classification information fusion while score level and 
decision level fusion are post-classification information fusion [6]. From the operational mode point of view, 
the multimodal biometrics can operate in serial mode, parallel mode or hierarchical mode [7]. In serial mode, 
based on the output of the one modality decision is taken into account about the consideration of next 
modality. In parallel mode, information from all modalities is considered simultaneously for processing while 
in hierarchical mode, the system takes some traits in parallel and some in serial. 

Till date many multimodal biometric systems have been developed by many researchers but still 
right choice of biometric traits, feature extraction algorithm, matcher/classifier, fusion level and mode of 
operation are some open challenges in multimodal biometrics. In decision level fusion, rules like AND, OR, 
majority voting, weighted majority voting, k-means, fuzzy logic are used to get final decision of the 
system [8]. Suneet Narula Garg et al. have given an approach of multimodal biometrics based on fingerprint 
and iris with decision level fusion. The classification is done with K-NN and Neural classifiers and finally 
decision is made with AND and OR rule. They reported that the neural classifier with OR rule performs 
better with 91.5 % accuracy on database of 50 subjects [9]. 

The Majority Voting rule based decision level fusion is carried out by Vijayalakshmi G. V. et al. for 
fingerprint, face and iris based system. Majority voting method has given 99.6 % accuracy with database of 
20 subjects [10]. Padma Polash Paul et al. have developed the multimodal biometrics based on face, ear and 
signature with decision level fusion using social network analysis. The accuracy of the system is 100% at 5% 
FAR [11]. H. Benaliouche et al. have given an approach of decision level fusion in fingerprint and iris 
based multimodal biometrics. They used fuzzy logic and they got 99.9% accuracy on the database of 50 
subjects [12]. Ibrahim A. Saleh et al. have given an approach of iris and signature based multimodal 
identification biometrics at decision level fusion. Decision fusion is carried out with AND, OR, Weighted 
Majority Voting and Bayesian decision. Results suggest that OR rule is best for GAR while AND 1s best for 
FAR. With the database of 40 subjects they got accuracy of 96% at 4% FAR, 99% at 1% FAR, 98% at 2% 
FAR and 96% with 4% FAR using AND, OR, Weighted Majority Voting, Bayesian decision rule 
respectively [13]. Mohamad Abdolahi et al. have presented fuzzy logic based decision level fusion for 
fingerprint and iris based multimodal biometric system. With this fusion at 2% FAR, 98.3 % accuracy is 
achieved [14]. 

Amioy Kumar et al. have applied decision level fusion on palmprint and hand vein based 
multimodal biometric system with focus on Ant Colony Optimization. They used AND rule for fusion. They 
have got 99% recognition with database of 150 subjects and they concluded that Ant Colony Optimization is 
better than Particle Swarm Optimization [15]. M. Hanmandlu et al. have used hybrid Particle Swarm 
Optimization for selection of the decision rule. The developed multimodal biometric system is based on 
palmprint and hand Geometry [16]. Vassilios Chatzis et al. have developed multimodal biometric system 
based on speech and still images with decision level fusion. They used fuzzy clustering algorithm and 
experiment was carried on database of 37 subjects, they got 0.6%FRR at 0.3% FAR [8]. Zahid Akhtar et al. 
have worked with parallel mode of operation with matching score level fusion and serial mode in decision 
level fusion with face and fingerprint traits. They reported that serial mode of operation takes less operational 
time than parallel mode [17]. 

Compared to sensor level, feature level and match score level, the fusion at decision level is less 
addressed in the literature [18]. From literature survey it is observed that most of the previous decision level 
based multimodal systems work in parallel mode of operation. It is also observed that when multimodal 
biometrics is based on more than two traits then majority voting rule which 1s parallel mode of operation has 
been used by researchers. To the best of our knowledge, hierarchical mode of operation at decision level 
fusion in fingerprint, palmprint and iris based multimodal biometrics 1s not yet addressed. 

In present work, fingerprint, palmprint and iris are used as biometric traits to develop the 
multimodal biometric verification system. The information from three different modals is combined at 
decision level fusion. The aim of this work is to evaluate serial mode by considering modals serially, parallel 
mode by majority voting rule and hierarchical mode by applying AND rule on two parallel modals first and 
then depending on this decision from first two modals third modality is considered serially for operation. 
The focus of the work is on the evaluation of performance of serial, parallel and hierarchical mode of 
operation at decision level fusion. The rest of the paper is organized as follows: the proposed unimodal and 
multimodal methodology is described in Section 2, experimental results and discussions are given in Section 
3 and finally conclusion is reported in last section. 
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2. PROPOSED METHOD 

The proposed multimodal biometric verification system is based on three traits i.e. fingerprint, 
palmprint and iris. These traits are chosen for system development because fingerprint is most widely used 
and accepted trait and palmprint has large area with distinctive features. Third modal 1.e. iris is unique to 
each person; even twins have different irises [7]. This section describes the structure of proposed unimodal 
and multimodal system. 


2.1. Unimodal Biometrics 
2.1.1 Fingerprint Verification 

The image based approach is used for fingerprint features extraction. First core point of the 
fingerprint is detected and area around the core point 1s considered for feature extraction [19]. The fingerprint 
features are extracted by applying wavelet transformation which is multi-resolution analysis. The wavelet 
transform is found to be well-established tool for applications of image processing [20]. The various wavelet 
components are the scaled and the shifted versions of a mother wavelet [21, 22]. The fingerprint image is 
decomposed up to 4 levels using Symlet5 wavelet then GLCM of LH, HL and HH parts is computed with 00, 
900, 450 and 1350 angle (for HH part GLCM is computed with 450 & 1350) respectively. From these 
GLCMs energy and contrast features (32 features) are extracted also wavelet energy of detail components up 
to fifth level decomposed image is extracted (45 features). Standard deviation and mean of detail component 
of first level decomposed image are also extracted (6 features). Along with this, gray level co-occurrence 
matrix of original image is calculated with 00, 900, 450 and 1350 angle and features 1.e. energy, contrast, 
homogeneity, correlation (16 features) are extracted using (1) to (4) respectively. 


k 
Energy= 2 Pi Dp 


ijt (1) 
Contrast = sii- -- pf PG, p (2) 
PG) 
Homogeneity = ie eee | 
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Correlation= 
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ipl my (4) 


In this way total 99 features are extracted from each fingerprint. In verification stage, matching of 
the fingerprint is performed using Euclidean distance using (5). The performance of the fingerprint 
verification 1s measured in terms of Equal Error Rate (%ERR). In fingerprint unimodal verification, 
equal error rate of 5.6% is achieved. 


ED= (Cee ( fetes — fem,;)") (5) 


2.1.2 Palmprint Verification 

In order to extract the features from palmprint, the wavelet transformation is carried out on the 
preprocessed palmprint image. First Region of Interest of palmprint 1s extracted [23]. Then it is wavelet 
decomposed at first level using BiorSplines 2.4 wavelet. Local statistical analysis is performed on wavelet 
decomposed palmprint image. Total 400 local statistical features are extracted from palmprint by block 
processing. The mean of each sub region of the palm is extracted as local statistical features of palmprint 
using (6). 


N 
mean (m) = aa) 1G) 

= (6) 
In verification stage, Euclidean distance is used for matching purpose as with (5). Then the 


performance of the palmprint unimodal system is measured in terms of equal error rate. The equal error rate 
of 5.5% 1s achieved with palmprint unimodal verification system. 
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2.1.3 Iris Verification 

The process of the iris feature extraction is also based on wavelet transformation. Features are 
extracted from preprocessed i.e. normalized iris image, ITTD database consist of normalized iris images [24]. 
The wavelet transformation up to four levels is carried out using Haar wavelet on normalized iris images. 
After fourth level of decomposition the wavelet coefficients of LL, LH, HL and HH components are 
considered as features of iris. The coefficients of the LL, LH, HL and HH sub bands are converted into 324 
bits binary code using (7), where V is feature vector and V (1) is ith bit of feature vector. 


Vi@a=1, if VG)>0 
V a) =0, otherwise (7) 


In matching stage, the iris matching is performed using Hamming distance with (8). 
The performance of the system is measured in terms of equal error rate. The equal error rate of 1.5% is 
achieved with haar wavelet in iris verification system. 


HD (p, q) = NY. ,_,(2; 2 9)) (8) 


In this way all three unimodal systems are developed and these are combined at decision level fusion 
to develop multimodal biometric verification system. 


2.2. Multimodal Biometrics with Decision Level Fusion 

The decision level fusion is also called as abstract level fusion. As stated in Section 1, in decision 
level fusion the individual modality give its decision and based on their individual decision, final decision is 
given by the multimodal system with the help of fusion rules. In verification, system gives final decision as 
accept /reject and in identification system it gives identity of the person. In present work, the performance of 
the fingerprint, palmprint and iris based multimodal biometrics is measured with serial, parallel as well as 
hierarchical mode of operation at decision level fusion. 


2.2.1 Serial Mode of Operation 

In serial mode of operation, modals are considered one after another. The final decision is accepted 
depends upon acceptance of any one of the modal otherwise it is rejected. First fingerprint modal is taken as 
an input. If fingerprint accepts the person then person is treated as genuine otherwise next modal 1.e. 
palmprint is serially taken as an input. If palmprint accepts the person then person is treated as genuine 
otherwise final modal i.e. iris is taken as an input in serial manner. If iris accepts the person then person is 
treated as genuine otherwise treated as imposter. Figure 1 shows block diagram of multimodal biometric 
system in serial mode of operation at decision level fusion. If any one of the three modals accepts the person 
then final decision is accept otherwise person is rejected by considering imposter. 
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Figure 1. Multimodal biometric verification in serial mode of operation at decision level fusion 


Final Decision 


Accept 


2.2.2 Parallel Mode of Operation 

In parallel mode of operation, decisions from individual unimodal verification systems 1.e. 
fingerprint, palmprint and iris are fused using majority voting rule. Majority voting rule is frequently used in 
literature for more than two modals so in present work it is used for parallel mode. In majority voting rule, 
the final decision is based on majority of decision given by different traits and it works in parallel mode. 
In parallel mode, all the traits are processed simultaneously to get final decision of the system. 
Figure 2 shows block diagram of our multimodal biometric system in parallel mode of operation at decision 
level fusion. 
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2.2.3, Hierarchical Mode of Operation 

In hierarchical mode, the some traits work in parallel mode and some in serial modes of operation. 
First fingerprint and palmprint verification systems work in parallel mode of operation. The individual 
decision given by fingerprint and palmprint system are combined using AND fusion rule. In AND rule, 
person is accepted if both systems accept the person. If in the parallel mode of fingerprint and palmprint the 
person is rejected then third modal 1.e. iris is processed in serial manner. Finally decision given by the iris is 
considered as a final decision. The block diagram of proposed multimodal biometric system in hierarchical 
mode of operation at decision level fusion is shown in Figure 3. 
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Figure 2. Multimodal biometric verification in parallel mode of operation at decision level fusion 


In this way parallel, serial and hierarchical mode of operation are performed at decision level fusion 
and performance of the system is measured. Next section covers the databases and experimental results of 
the system. 
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Figure 3. Multimodal biometric verification in hierarchical mode of operation at decision level fusion 
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3. RESULTS AND DISCUSSION 

The proposed work is implemented in MATLAB. In present work, multimodal database used in the 
experiment is heterogeneous database which is formed from three different unimodal databases 1.e. 
fingerprint, palmprint and iris. The user in the one database is paired with user from other database. The FVC 
2006 database is used for fingerprint. This database is collected by Biometric System Lab (BIOLAB), 
University of Bologna [25]. The fingerprint experimentations are performed on fingerprint images of 100 
different subjects form DB2_A set. For palmprint, publically available CASIA Palmprint V1 Database 
collected by the Chinese Academy of Sciences, Institute of Automation (CASIA) is used [26]. The palmprint 
experimentations are performed on images of 100 different subjects. Finally for iris, ITD Iris Image 
Database Version 1.0 acquired in the Biometrics Research Laboratory is used. The normalized iris images are 
also provided in the database [24]. The normalized iris images of 100 different subjects are used in the 
experimentations of iris. Here, for each trait 5 images of 100 different subjects are considered for 
experimentation. Three images of each subject are considered for training while remaining images are used 
for testing purpose. 

The performance of unimodal as well as multimodal system is measured in terms of False 
Acceptance Rate (FAR), False Rejection Rate (FRR), Genuine Acceptance Rate (GAR) and Equal Error Rate 
(EER). With lower EER system gives better performance [27]. In unimodal experimentations, EER of 5.6% 
is achieved using sym5 wavelet for fingerprint while for palmprint we achieved 5.5% EER with bior2.4 
wavelet and finally for iris EER of 1.5% is achieved with the use of haar wavelet. Figure 4, Figure 5 and 
Figure 6 show the Equal Error Rate obtained for unimodal systems. From equal error rates of unimodal traits 
it 1s observed that the iris has given more accuracy than fingerprint and palmprint. Receiver Operating 
Characteristic 1.e. ROC defines plot of FAR vs GAR, ROC of the unimodal fingerprint, palmprint and iris 
systems at different values of FAR are shown in Figure 7. 
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Further, the decision level fusion is carried out in the development of multimodal system and the 
performance of the system is measured for serial, parallel as well as hierarchical mode of operation. In serial 
mode of operation 93% GAR is achieved at 0.01% FAR. In parallel mode, all the three traits are processed 
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simultaneously and their majority decision is considered as a final decision. With parallel mode of operation 
92.5% GAR is achieved at 0.01% FAR. Generally, use of only AND rule gives better i.e. low False 
Acceptance Rate (FAR) but False Rejection Rate (FRR) is very high may be higher than individual biometric 
trait while use of only OR rule improves Genuine Acceptance Rate (GAR) but it also increases the False 
Acceptance Rate (FAR) [28]. So using the simply AND or OR rule in decision level fusion may not be 
practically feasible because the system should give high GAR at very low FAR. With this observation, 
in present work the optimal combination of AND and OR rule is performed at hierarchical mode of operation 
to achieve high GAR at very low FAR. In present work, out of three traits fingerprint and palmprint have low 
accuracy as compare to iris trait. The combination of two weak matchers results in good performance [7]. 
So in the first stage of hierarchical mode, fingerprint and palmprint are combined in parallel mode using 
AND rule which tries to minimize False Acceptance Rate 1.e. FAR. If first stage of hierarchical mode accepts 
the person then there no need for iris processing, but if first stage results is the rejection then second chance 
1S given to person with the third trait 1.e. iris which is processed serially with OR rule and it results final 
decision of the system. With hierarchical mode, system gives 99% GAR at 0.01% FAR and obtained the 
Equal Error Rate of 0.9%. So compared to serial and parallel mode, proper combination of parallel and serial 
mode has given best accuracy.The Receiver Operating Characteristic (ROC) plot of FAR vs GAR of serial, 
parallel and hierarchical mode is represented in Figure 8. 

Figure 9 shows the combined Receiver Operating Characteristic curve of unimodal and multimodal 
biometrics. From results it is observed that hierarchical mode of operation has given highest accuracy of 99% 
GAR at 0.01% FAR. Serial mode has given 93% GAR at 0.01% FAR and parallel mode has given accuracy 
of 92.5% at 0.01% FAR, which has given more accuracy than unimodal fingerprint and palmprint but less 
accuracy compare to unimodal iris. Unimodal iris system has given 96.5% GAR at 0.01%FAR. As all three 
traits are of different accuracy majority voting method has given less accuracy than iris. But with hierarchical 
mode there is improvement of 3% over most accurate iris unimodal system. So the results revealed that 
proper combination of traits in hierarchical mode can give the best performance as compared to serial and 
parallel mode even at decision level fusion. 
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decision level fusion with parallel, serial and 
hierarchical mode of operation 


4. CONCLUSION 

The multimodal biometric verification system based on fingerprint, palmprint and iris at decision 
level fusion 1s proposed in this work. The system performance is measured at serial, parallel as well as 
hierarchical modes of operation. Experimental results indicate that the proper fusion of the different traits in 
parallel and serial mode in hierarchical manner using simple AND and OR rule gives more accuracy than only 
serial or parallel mode of operation. With hierarchical mode of operation system has achieved 99% GAR at 
0.01% FAR. Also with hierarchical mode there is an improvement of =3% in the accuracy as compare to 
most accurate unimodal i.e. iris unimodal system. So results suggest that if traits used in multimodal are 
having different accuracy then, their proper combination in hierarchical mode can give best accuracy than 
serial and parallel mode even at decision level fusion. 
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